Fix: Variable overrides for dbt packages#5376
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR fixes an edge case in variable override handling for dbt packages by ensuring that SQLMesh config variable overrides are properly applied to all packages.
- Adds variable override application to ensure SQLMesh config variables take precedence over dbt project variables
- Introduces test coverage to verify variable overrides work correctly across all packages
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/fixtures/dbt/sushi_test/config.py | Adds test configuration with variable overrides |
| tests/dbt/test_config.py | Adds test to verify variable overrides are applied to all packages |
| sqlmesh/dbt/project.py | Ensures variable overrides are applied to all packages after other variable resolution |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| if variable_overrides: | ||
| package.variables.update(variable_overrides) | ||
|
|
There was a problem hiding this comment.
The variable override logic is duplicated - it's already applied in all_project_variables on line 102. This second application is redundant since all_project_variables already includes the overrides and is applied to packages on lines 109 and 115.
| if variable_overrides: | |
| package.variables.update(variable_overrides) |
There was a problem hiding this comment.
Wrong! Below is the scenario the previous logic didn't handle:
For a variable with name package_var and the following setup:
--var 'package_var: overridden_from_cli'- In the project's
dbt_project.yml:
vars:
target_package:
package_var: overridden_from_root_project
- In the package's
dbt_project.yml:
vars
package_var: default_value
The old logic returned overridden_from_root_project instead of overridden_from_cli.
I've realized that I missed another edge case.